User interface with screen spanning icon morphing

ABSTRACT

Methods and apparatus for indicating a status of an application that is displayable on one or more displays of a handheld computing device. An icon may be provided that indicates the status and/or potential statuses of the application (e.g., whether the application is expandable and/or expanded). The icon may be changeable between a first state and a second state depending on the status of the application. The change in the icon from the first state to the second state may be animated along with an animated change of the application between display states. As such, a user may observe the icon to determine the status of the application with respect to the one or more displays (e.g., whether the application is expandable, expanded, or expanding).

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 61/389,000, filed Oct. 1, 2010, entitled “DUAL DISPLAY WINDOWINGSYSTEM”; Provisional Application Ser. No. 61/389,117, filed Oct. 1,2010, entitled “MULTI-OPERATING SYSTEM PORTABLE DOCKETING DEVICE”; andProvisional Application Ser. No. 61/389,087, filed Oct. 1, 2010,entitled “TABLET COMPUTING USER INTERFACE”. Each and every part of theforegoing provisional applications is hereby incorporated by referencein their entirety.

BACKGROUND

As the computing and communication functions of handheld computingdevices become more powerful, the user interface and display elements ofsuch devices have evolved by attempting to adapt user interface regimesdeveloped for personal computers for use with handheld computingdevices. However, this attempt to adapt prior user interface regimes hasbeen met with various hurdles.

For instance, the majority of current handheld computing devices makeuse of a physical keypad for user interface. Many differentimplementations of physical keypads exist that vary in orientation andrelationship to the device screen. However, in every case the physicalkeypads take up a certain percentage of the physical space of the deviceand increase the weight of the device. In addition to the disadvantagesof size and weight, physical keypads are not configurable in the samemanner as a touch screen based user interface. While certain limitedforms of physical keypads currently have, on the keys themselves,configurable displays, such as eInk or OLED surfaces, to allow forreconfiguration of the keys, even in these cases, the physical layout ofkeys is not modifiable. Rather, only the values associated with thephysical keys on the keypad may be changed.

Other methods may provide increased user configurability of physicalkeypads. These methods may include stickers and/or labels that can beadded to keys to reference modified functions or plastic overlays on topof the keypad denoting different functional suites. For instance, theZBoard keyboard, meant for laptop or desktop computer use, incorporatesa dual layered physical keyboard which separates the keys and theirlayout from the connections which send signals to the machine. As such,different physical keyboard inserts for different applications can beinserted into a holder allowing full configurability such that theorientation and layout of the keys in addition to their denotation offunction is configurable. This model could be extended to handheldcomputing devices; however, the rate at which such a modular keypad canchange functions is much slower than a touch screen user interface.Furthermore, for each potential functional suite, an additional physicalkey layout must be carried by the user, greatly increasing the overallphysical size and weight of such implementations. One advantage of aphysical keypad for handheld computing devices is that the user inputspace is extended beyond the user display space such that none of thekeys themselves, the housing of the keys, a user's fingers, or apointing device obscure any screen space during user interfaceactivities.

A substantial number of handheld computing devices make use of a smalltouch screen display to deliver display information to the user and toreceive inputs from the user. In this case, while the configurability ofthe device may be greatly increased and a wide variety of user interfaceoptions may be available to the user, this flexibility comes at a price.Namely, such arrangements require shared screen space between thedisplay and the user interface. While this issue is shared with othertypes of touch screen display/user interface technology, the small formfactor of handheld computing devices results in a tension between thedisplayed graphics and area provided for receiving inputs. For instance,the small display further constrains the display space, which mayincrease the difficulty of interpreting actions or results while akeypad or other user interface scheme is laid overtop or to the side ofthe applications in use such that the application is squeezed into aneven smaller portion of the display. Thus a single display touch screensolution, which solves the problem of flexibility of the user interfacemay create an even more substantial set of problems of obfuscation ofthe display, visual clutter, and an overall conflict of action andattention between the user interface and the display.

Single display touch screen devices thus benefit from user interfaceflexibility, but are crippled by their limited screen space such thatwhen users are entering information into the device through the display,the ability to interpret information in the display can be severelyhampered. This problem is exacerbated in several key situations whencomplex interaction between display and interface is required, such aswhen manipulating layers on maps, playing a game, or modifying datareceived from a scientific application. This conflict between userinterface and screen space severely limits the degree to which the touchbased user interface may be used in an intuitive manner.

SUMMARY

A first aspect includes a method for controlling a handheld computingdevice including one or more displays. The method includes displaying anapplication in a first display condition. The first display conditionincludes an icon in a first state corresponding to the first displaycondition. The method further includes receiving a gesture input at agesture sensor. For instance, the gesture sensor may be a touchsensitive device, optical device (e.g., a camera), or other sensoroperable to detect user inputs in the form of a gesture. The method alsoincludes modifying the manner in which the application is displayed inresponse to the receiving step. The modifying step includes changing theapplication to a second display condition such that the icon is in asecond state corresponding to a second display condition.

A second aspect includes a handheld computing device comprising aprocessor. The device also includes a first display operable to displaya first screen. The first display is in operative communication with theprocessor. The device also includes a second display that is operable todisplay a second screen. The second display is also in operativecommunication with the processor. The device also includes at least onegesture sensor operable to receive a gesture input. The processor isoperable to execute an application that is displayable on at least oneof the first display and the second display in response to the gestureinput. The application includes an icon indicative of a display statusof the application with respect to the first and second displays.

A number of feature refinements and additional features are applicableto the first and second aspects. These feature refinements andadditional features may be used individually or in any combination. Assuch, each of the following features that will be discussed may be, butare not required to be, used with any other feature or combination offeatures of any of the aspects presented herein.

In one embodiment, the icon may be changed in proportion to the relativeposition of the application with respect to the first and seconddisplay. The first display condition may include displaying theapplication in only a first display. In this regard, the modifying mayinclude expanding the application to occupy at least a first and seconddisplay. Additionally or alternatively, the first display condition mayinclude displaying the application in at least a first and seconddisplay. In this regard, the modifying may include minimizing theapplication to occupy only one of the first and second displays.

The first and second dimension may define a first aspect ratio, and thethird and fourth dimension may define a second aspect ratio. In oneembodiment, the first aspect ratio may be different than the secondaspect ratio. In this regard, the first dimension and the thirddimension may be equal. In one embodiment, the first dimension and thethird dimension may be vertical dimensions. In turn, the seconddimension may correspond to a width of the application when displayed inthe first display, and the fourth dimension may correspond with thewidth of the application when displayed in the first and the seconddisplay.

In another embodiment, the modifying may include animating the change ofthe application. As such, the animating may include the change of theicon from the first aspect ratio to the second aspect ratio. The iconmay be indicative of whether the application is expandable, whether theapplication is expanded, or whether the application is expanding.

In yet another embodiment, the handheld computing device may be a smartphone. The first display and second display may be positionable withrespect to each other between an open and closed position. In thisregard, when in the open position, the first display and the seconddisplay, may be concurrently visible from the vantage point of a user.In contrast, when in the closed position, only one of the first displayand the second display may be visible from the vantage point of a user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of an embodiment of a handheld computingdevice.

FIGS. 2A-D are graphical representations of an embodiment of a handheldcomputing device in various instances of operation.

FIGS. 3A-K are graphical representations of an embodiment of a handheldcomputing device provided in different positions, orientations, andinstances of operation.

FIG. 4 includes graphical representations of various gesture inputs forcontrolling a handheld computing device.

FIGS. 5A-5C are graphical representations of an embodiment of a handheldcomputing device during a progression wherein a screen is expanded intoa second display.

FIGS. 6A-6B are schematic views of embodiments of a handheld computingdevice provided with touch sensitive devices.

FIG. 7 is a graphical representation of an embodiment of a gestureinput.

DETAILED DESCRIPTION

The present disclosure is generally related to gesture inputs forinteraction with a computing device. The interface controls areparticularly suited for control of devices that have one or moredisplays capable of displaying graphical user interfaces (GUIs) on ahandheld portable device. The following disclosure may, in variousembodiments, be applied to other computing devices capable of displayingand responding to a GUI (e.g., laptop computers, tablet computers,desktop computers, touch screen monitors, etc.) and is not intended tobe limited to handheld computing devices unless otherwise explicitlyspecified.

FIG. 1 depicts an embodiment of a handheld computing device 100. Thehandheld computing device 100 may include a first display 102 and asecond display 104. Additionally, while two displays (102, 104) may beshown and described below with regard to the functionality of variousembodiments of handheld computing devices, a handheld computing devicemay be provided that includes one or more displays. In any regard, thefirst display 102 and the second display 104 may be independentlycontrollable. The displays may be operative to display a displayed imageor “screen”. As used herein, the term “display” is intended to connotedevice hardware, whereas “screen” is intended to connote the displayedimage produced on the display. In this regard, a display is a physicalhardware that is operable to render a screen. A screen may encompass amajority of the display. For instance, a screen may occupy substantiallyall of the display area except for areas dedicated to other functions(e.g., menu bars, status bars, etc.) A screen may be associated with anapplication and/or an operating system executing on the handheldcomputing device 100. For instance, application screens or desktopscreens may be displayed. An application may have various kinds ofscreens that are capable of being manipulated as will be describedfurther below. In an embodiment, each display may have a resolution of480 pixels by 800 pixels, although higher and lower resolution displaysmay also be provided.

A screen may be associated with an operating system, an application, orthe like. In some instances, a screen may include interactive features(e.g., buttons, text fields, toggle fields, etc.) capable ofmanipulation by way of a user input. The user input may be received byvarious input devices (e.g., a physical keyboard, a roller ball,directional keys, a touch sensitive device, etc.). In some instances, ascreen may simply include graphics and have no ability to receive aninput by a user. In other instances, graphics features and inputfeatures may both be provided by a screen. As such, the one or moredisplays of a handheld computing device, the screens displayed on theone or more displays, and various user input devices may comprise a GUIthat allows a user to exploit functionality of the handheld computingdevice.

The handheld computing device 100 may be configurable between a firstposition and a second position. In the first position, a single display(e.g., the first display 102 or the second display 104) may be visiblefrom the perspective of a user. Both displays 102, 104 may be exposed onan exterior of the handheld device 100 when in the first position, butthe displays 102, 104 may be arranged in a non-adjacent manner such thatboth displays 102, 104 are not concurrently visible from the perspectiveof a user (e.g., one display may be visible from the front of the device100 and the other display may be visible from the back of the device100).

The handheld computing device 100 may also be provided in the secondposition such that the displays 102, 104 may be concurrently viewablefrom the perspective of a user (e.g., the displays 102, 104 may bepositioned adjacent to one another). The displays 102, 104 may bedisplayed in the second position such that the displays 102, 104 arearranged end-to-end or side-by-side. Additionally, the displays 102, 104may be arranged in a portrait orientation or a landscape orientationwith respect to a user. As will be discussed further below, a portraitorientation is intended to describe an arrangement of the handheldcomputing device, wherein the longer dimension of the display of thehandheld computing device is vertically oriented (e.g., with respect togravity or the perspective of a user). A landscape orientation isintended to describe an arrangement wherein the shorter dimension of thedisplay of the handheld computing device is vertically oriented (e.g.,with respect to gravity or the perspective of a user). Furthermore, thelonger dimension and shorter dimension may refer to each displayindividually or the combined viewing area of the one or more displays ofthe device. Thus, when the individual displays are arranged in aportrait orientation, the overall display area may be arranged in alandscape orientation, and vice versa. Additionally, the displays andscreens may be in different respective orientations. For instance, whenthe displays are in a landscape orientation, one or more screens may berendered in a portrait orientation on the displays or vice versa.

The handheld computing device 100 may be manipulated between the firstposition (i.e., a single display visible from a user's perspective) andthe second position (i.e., at least two displays concurrently visiblefrom the user's perspective) in a variety of manners. For instance, thedevice 100 may include a slider mechanism such that the first and seconddisplays 102, 104 are disposable adjacent to one another in a parallelfashion in a second position and slideable to the first position whereonly a single display is viewable and the other display is obscured bythe viewable display.

Alternatively, the device 100 may be arranged in a clam shell typearrangement wherein a hinge is provided between the first display 102and the second display 104 such that the displays 102, 104 areconcurrently visible by a user when in the second position (i.e., anopen position). The displays 102, 104 may be provided on an interiorclam shell portion or an exterior clam shell portion of the device 100.In this regard, both displays 102, 104 may be visible from the front andthe back of the device, respectively, when the device is in the firstposition (i.e., the closed position). When the device 100 is in the openposition, the displays 102, 104 may be provided adjacent and parallel toone another. Alternative arrangements of the handheld computing device100 are contemplated wherein different arrangements and/or relativelocations of the displays may be provided when in the first and secondposition.

In addition, the first display 102 and the second display 104 may beprovided as entirely separate devices. In this regard, a user maymanipulate the displays 102, 104 such that they may be positionedadjacent to one another (e.g., side-by-side or end-to-end). The displays102, 104 may be in operative communication when adjacently positionedsuch that the displays 102, 104 may operate in the manner provided ingreater detail below when adjacently positioned (e.g., via physicalcontacts, wireless communications, etc.). A retention member (not shown)may be provided to retain the separate displays 102, 104 in an adjacentposition. For instance, the retention member may include coordinatingmagnets, mechanical clips or fasteners, elastic members, etc.

While the foregoing has referenced two displays 102 and 104, alternateembodiments of a handheld device may include more than two displays. Inthis regard, the two or more displays may behave in a manner inaccordance with the foregoing wherein only a single display is viewableby a user in a first position and multiple displays (i.e., more than twodisplays) are viewable in a second position. Additionally, in oneembodiment, the two displays 102 and 104 may comprise separate portionsof a unitary display. As such, the first display 102 may be a firstportion of the unitary display and the second display 104 may be asecond portion of the unitary display. For instance, the handheldcomputing device 100 (e.g., having a first and second display 102 and104) may be operatively connected to the unitary display (e.g., via aconnector or a dock portion of the unitary display) such that the firstdisplay 102 and the second display 104 of the handheld computing device100 are emulated on the unitary display. As such, the unitary displaymay have first and second portions corresponding to and acting in asimilar manner to the first and second display 102 and 104 of thehandheld computing device 100 described below.

The handheld computing device 100 may further include one or more inputdevices that may be used to receive user inputs. These input devices maybe operative to receive gesture inputs from a user, and, accordingly,may be referred to generally as gesture sensors. A number of differenttypes of gesture sensors may be provided. Some examples include, but arenot limited to traditional input devices (keypads, trackballs, etc.),touch sensitive devices, optical sensors (e.g., a camera or the like),etc. The discussion contained herein may reference the use of touchsensitive devices to receive gesture inputs. However, the use of touchsensitive devices is not intended to limit the means for receivinggesture inputs to touch sensitive devices alone and is provided forillustrative purposes only. Accordingly, any of the foregoing means forreceiving a gesture input may be used to produce the functionalitydisclosed below with regard to gesture inputs received at touchsensitive devices.

In this regard, the handheld computing device 100 may include at least afirst touch sensor 106. Furthermore, the handheld computing device mayinclude a second touch sensor 108. The first touch sensor 106 and/or thesecond touch sensor 108 may be touchpad devices, touch screen devices,or other appropriate touch sensitive devices. Examples includecapacitive touch sensitive panels, resistive touch sensitive panels, ordevices employing other touch sensitive technologies. The first touchsensor 106 and/or second touch sensor 108 may be used in conjunctionwith a portion of a user's body (e.g., finger, thumb, hand, etc.), astylus, or other acceptable touch sensitive interface mechanisms knownin the art. Furthermore, the first touch sensor 106 and/or the secondtouch sensor 108 may be multi-touch devices capable of sensing multipletouches simultaneously.

The first touch sensor 106 may correspond to the first display 102 andthe second touch sensor 108 may correspond to the second display 104. Inone embodiment of the handheld computing device 100, the first display102 and the first touch sensor 106 comprise a first touch screen display110. In this regard, the first touch sensor 106 may be transparent ortranslucent and positioned with respect to the first display 102 suchthat a corresponding touch received at the first touch sensor 106 may becorrelated to the first display 102 (e.g., to interact with a screenrendered on the first display 102). Similarly, the second display 104and the second touch sensor 108 may comprise a second touch screendisplay 112. In this regard, the second touch sensor 108 may bepositioned with respect to the second display 104 such that a touchreceived at the second touch sensor 108 may be correlated to the seconddisplay 104 (e.g., to interact with a screen rendered on the seconddisplay 104). Alternatively, the first touch sensor 106 and/or thesecond touch sensor 108 may be provided separately from the displays102, 104. Furthermore, in an alternate embodiment, only a single gesturesensor may be provided that allows for inputs to control both the firstdisplay 102 and the second display 104. The single gesture sensor mayalso be provided separately or integrally with the displays.

In this regard, the first and second touch sensors 106, 108 may have thesubstantially same footprint on the handheld computing device 100 as thedisplays 102, 104. Alternatively, the touch sensors 106, 108 may have afootprint including less of the entirety of the displays 102, 104.Further still, the touch sensors 106, 108 may include a footprint thatextends beyond the displays 102, 104 such that at least a portion of thetouch sensors 106, 108 are provided in non-overlapping relation withrespect to the displays 102, 104. As discussed further below, the touchsensors 106, 108 may alternatively be provided in completenon-overlapping relation such that the footprint of the touch sensors106, 108 is completely different than the footprint of the displays 102,104.

With reference to FIGS. 9A-B, various potential arrangements aredepicted for the first display 102, the second display 104, and touchsensors 106′, 106″, and 108″. In FIG. 9A, the first 102 and seconddisplay 104 are arranged side-by-side such that a crease 196 separatesthe displays. In this regard, the first display 102 and second display104 may be arranged in a clam-shell type arrangement such that thecrease 196 includes a hinge that allows for pivotal movement between thefirst display 102 and second display 104 as discussed above. A touchsensor 106′ may span the width of both the first display 102 and thesecond display 104. In this regard, the touch sensor 106′ may span thecrease 196 without interruption. Alternatively, as shown in FIG. 9B,separate touch sensors 106″ and 108″ may be provided on either side ofthe crease 196. In this regard, each of the touch sensors 106″ and 108″may span the width of each of the first display 102 and second display104, respectively.

In any of the arrangements shown in FIGS. 9A-B, the displays (102, 104)may also comprise touch screen displays that may be used in conjunctionwith touch sensitive portions that are provided separately from thetouch screen displays. Thus, displays 102 and 104 may both comprisetouch screen displays and be provided in addition to touch sensitivedevices 106′, 106″, and 108″. Accordingly, a combination of touch screendisplays (e.g., 110, 112) and off display touch sensors (e.g., 106′,106″, 108″) may be provided for a single device. Touch inputs may bereceived at both a touch screen display (110, 112) and off display touchsensor (106′, 106″, 108″). In this regard, gestures received at an offscreen display sensor may have a different functionality than the samegesture received at a touch screen display. Also, a touch sensitivedevice may be divided into a plurality of zones. The same gesturereceived in different zones may have different functionality. Forinstance, a percentage (e.g., 10%, 25%, etc.) of the touch sensitivedevice at the top or bottom of the display may be defined as a separatezone than the remainder of the touch sensitive device. Thus, a gesturereceived in this zone may have a different functionality than a gesturereceived in the remainder of the touch sensitive device.

The handheld computing device 100 may further include a processor 116.The processor 116 may be in operative communication with a data bus 114.The processor 116 may generally be operative to control thefunctionality of the handheld device 100. For instance, the processor116 may execute an operating system and be operative to executeapplications. The processor 116 may be in communication with one or moreadditional components 120-134 of the handheld computing device 100 aswill be described below. For instance, the processor 116 may be indirect communication with one more of the additional components 120-134or may communicate with the one or more additional components 120-134via the data bus 114. Furthermore, while the discussion below maydescribe the additional components 120-134 being in operativecommunication with the data bus 114, in other embodiments any of theadditional components 120-134 may be in direct operative communicationwith any of the other additional components 120-134. Furthermore, theprocessor 116 may be operative to independently control the firstdisplay 102 and the second display 104 and may be operative to receiveinput from the first touch sensor 106 and the second touch sensor 108.The processor 116 may comprise one or more different processors. Forexample, the processor 116 may comprise one or more application specificintegrated circuits (ASICs), one or more field-programmable gate arrays(FPGAs), one or more general purpose processors operative to executemachine readable code, or a combination of the foregoing.

The handheld computing device may include a battery 118 operative toprovide power to the various devices and components of the handheldcomputing device 100. In this regard, the handheld computing device 100may be portable.

The handheld computing device 100 may further include a memory module120 in operative communication with the data bus 114. The memory module120 may be operative to store data (e.g., application data). Forinstance, the memory 120 may store machine readable code executable bythe processor 116 to execute various functionalities of the device 100.

Additionally, a communications module 122 may be in operativecommunication with one or more components via the data bus 114. Thecommunications module 122 may be operative to communicate over acellular network, a Wi-Fi connection, a hardwired connection or otherappropriate means of wired or wireless communication. The handheldcomputing device 100 may also include an antenna 126. The antenna 126may be in operative communication with the communications module 122 toprovide wireless capability to the communications module 122.Accordingly, the handheld computing device 100 may have telephonycapability (i.e., the handheld computing device 100 may be a smartphonedevice). An audio module 124 may also be provided in operativecommunication with the data bus 114.

The audio module 124 may include a microphone and/or speakers. In thisregard, the audio module 124 may be able to capture audio or producesounds. Furthermore, the device 100 may include a camera module 128. Thecamera module 128 may be in operative communication with othercomponents of the handheld computing device 100 to facilitate thecapture and storage of images or video.

Additionally, the handheld computing device 100 may include an I/Omodule 130. The I/O module 130 may provide input and output features forthe handheld computing device 100 such that the handheld computingdevice 100 may be connected via a connector or other device in order toprovide syncing or other communications between the handheld computingdevice 100 and another device (e.g., a peripheral device, anothercomputing device etc.).

The handheld computing device 100 may further include an accelerometermodule 132. The accelerometer module 132 may be able to monitor theorientation of the handheld computing device 100 with respect togravity. In this regard, the accelerometer module 132 may be operable todetermine whether the handheld computing device 100 is substantially ina portrait orientation or landscape orientation. The accelerometermodule 132 may further provide other control functionality by monitoringthe orientation and/or movement of the handheld computing device 100.

The handheld computing device 100 may also include one or more hardwarebuttons 134. The hardware buttons 134 may be used to control variousfeatures of the handheld computing device 100. The hardware buttons 134may have fixed functionality or may be contextual such that the specificfunction of the buttons changes during operation of the handheldcomputing device 100. Examples of such hardware buttons may include, butare not limited to, a volume control, a home screen button, an endbutton, a send button, a menu button, etc.

With further reference to FIGS. 2A-D, various screens of an embodimentof a device are shown. multiple screens may be shown, only one or asubset of the multiple screens may be shown on the displays of thedevice at any one moment. In this regard, a screen may be described in arelative location to the displays or other screens (e.g., to the left ofa display, to the right of a display, under another screen, aboveanother screen, etc.). These relationships may be logically establishedsuch that no physical display reflects the relative position. Forinstance, a screen may be moved off a display to the left. While thescreen is no longer displayed on the display, the screen may have avirtual or logical position to the left of the display from which it wasmoved. This logical position may be recognized by a user and embodied invalues describing the screen (e.g., values stored in memory correspondto the screen). Thus, when referencing screens in relative locations toother screens, the relationships may be embodied in logic and notphysically reflected in the display of the device.

FIGS. 2A-D may display a number of different screens that may bedisplayed at various instances of operation of a handheld device and arenot intended to be presented in any particular order or arrangement.Single screen applications and multi screen applications may beprovided. A single screen application is intended to describe anapplication that is capable of producing a screen that may occupy only asingle display at a time. A multi screen application is intended todescribe an application that is capable of producing one or more screensthat may simultaneously occupy multiple displays. Additionally, a multiscreen application may occupy a single display. In this regard, a multiscreen application may have a single screen mode and a multi screenmode.

A desktop sequence 136 is displayed in FIG. 2A. The desktop sequence 136may include a number of individual desktop screens 138 a-138 f. Thus,each desktop screen 138 may occupy substantially the entirety of asingle display (e.g., the first display 102 or second display 104 ofFIG. 1). The desktop screens 138 a-138 f may be in a predetermined ordersuch that the desktop screens 138 a-138 f appear consecutively and theorder in which the desktop screens appear may not be reordered. However,the desktop screens 138 a-138 f may be sequentially navigated (e.g., inresponse to a user input). That is, one or more of the desktop screens138 a-138 f may be sequentially displayed on a handheld device ascontrolled by a user input.

Additionally, FIG. 2B displays a hierarchal application sequence 140 ofa multi screen application. The hierarchal application sequence 140 mayinclude a root screen 142, one or more node screens 144, and a leafscreen 146. The root screen 142 may be a top level view of thehierarchical application sequence 140 such that there is no parentscreen corresponding to the root screen 142. The root screen 142 may bea parent to a node screen 144. One or more node screens 144 may beprovided that are related as parent/children. A node screen may alsoserve as a parent to a leaf screen 146. By leaf screen 146, it is meantthat the leaf screen 146 has no corresponding node screen 144 for whichthe leaf screen 146 is a parent. As such, the leaf screen does not haveany children node screens 144. FIG. 2C depicts various single screenapplications 148 a, 148 b, and 148 c arranged sequentially. Each ofthese single screen applications may correspond to a different executingapplication. For instance, in FIG. 2C Application 4, Application 5, andApplication 6 may be executing on the device and correspond to eachsingle screen 148 a, 148 b, and 148 c, respectively.

FIG. 2D also includes an empty view 166. The empty view 166 may be usedduring transitions of a screen (e.g., movement of screen between a firstdisplay and a second display). It is not necessary that the empty view166 be interpretable by the user as an effective GUI screen. The emptyview 166 merely communicates to the user that an action regarding thescreen (e.g., the movement of the screen with respect to one or moredisplays) is occurring. An application displaying an empty view 166 neednot be able to rest, wait, process or interpret input. The empty view166 may display a screen, or a representation thereof, as it is beingmoved in proportion to the amount of the screen that has been moved froma first display to a second display as will be discussed in greaterdetail below. In this regard, the empty view 166 may be used to relateinformation regarding the position of a screen during a transition ofthe screen (e.g., in response to gesture). An empty view 166 is onlyintended to refer to a screen not capable of receiving an input (e.g., ascreen in transition). In this regard, the display of an empty view 166may include an animation or the like showing the response of a screen asit is being moved or changed (e.g., modified into or out of a landscapemode).

FIGS. 3A-K depict various arrangements and statuses of displays 102, 104of a device that are possible in various embodiments of a handheldcomputing device according to the present disclosure. For instance, whenin the first (e.g., closed) position, a closed front display 168 may bevisible as shown in FIG. 3A. The closed front display 168 may correspondwith the first display 102 or the second display 104. The closed front168 as displayed may be occupied by a desktop screen D1 138 as shown inFIG. 3A. Alternatively, an application with a single screen or a multiscreen application in single screen mode may be displayed in the closedfront 168. A closed back display 170 may be viewable from an oppositeside of the display when the device is in a closed position, as shown inFIG. 3B. The closed back 170 may display a different desktop screen orapplication screen than the closed front 168 (e.g., as shown in FIG. 3H)or may simply display an empty view 166 (e.g., displaying an icon orother graphic) and lack functionality as an interface.

FIG. 3C depicts a closed device in a landscape orientation 172 a. In oneembodiment, a landscape mode (i.e., wherein the display is adjusted todisplay a screen in a landscape orientation) may not be enabled as shownin FIG. 3C. Alternatively, the landscape mode may be enabled such thatthe screen (e.g., application screen 148) is modified when the device issensed in a landscape orientation 172 b, such that the screen 148 isrendered in a landscape orientation as shown at FIG. 3D.

The device may further be provided in a second (e.g., open) position 174as shown in FIG. 3E. In the open position 174, at least two displays102, 104 are arranged such that the two displays 102, 104 are bothvisible from the vantage point of a user. The two displays 102, 104 maybe arranged in a side-by-side fashion when in the open position 174.Thus, each of the two displays 102, 104 may display separate screens.For instance, the displays 102, 104 may each display a separate desktopscreen 138 a, 138 b, respectively. While the individual displays 102 and104 are in a portrait orientation as shown in FIG. 3E, it may beappreciated that the full display area (comprising both the firstdisplay 102 and the second display 104) may be arranged in a landscapeorientation. Thus, whether the device as depicted in FIG. 3E is in alandscape or portrait orientation may depend on whether the displays arebeing used individually or collectively. If used collectively as aunitary display, the device may be in a landscape orientation, whereasif the displays are used separately, the orientation shown in FIG. 3Emay be referred to as a portrait orientation.

Additionally, when the device is in an open position 174 as shown inFIG. 3F, a similar dependency with regard to the use of the screens as aunitary display or separate displays may also affect whether the deviceis in a portrait orientation or landscape orientation. As can beappreciated, each individual screen is in a landscape orientation, suchthat if the displays are used separately, the device may be in alandscape orientation. If used as a unitary display, the device may bein a portrait orientation. In any regard, as shown in FIG. 3F, a singlescreen 148 may occupy a first display 102 and the second display 104 maydisplay a desktop screen 138. The single screen 148 may be displayed ina landscape or portrait mode. Alternatively, a device in an openorientation may display a multi screen application 156 that may occupyboth displays 102, 104 in a portrait orientation as shown in FIG. 3Gsuch that the individual displays are in a landscape orientation.

FIGS. 3I-K depict the potential arrangements of the screens of a multiscreen application 152. The multi screen application 152 may, in onemode, occupy a single display 102 when the device is in a closedposition 168 as shown in FIG. 3I. That is, the multi screen application152 may be in a single screen mode. Alternatively, when the device is inan open position as shown in FIG. 3J, the multi screen application 152may still occupy a single display 102 in single screen mode.Furthermore, the multi screen application 152 may be expanded to occupyboth displays 102, 104 when the device is in the open position as shownin FIG. 3K. In this regard, the multi screen application 152 may alsoexecute in a multi screen mode. Various options may be provided forexpanding the multi screen application 152 from a single screen mode toa multi screen mode.

For example, the multi screen application 152 may be maximized from asingle screen mode displayed in a single display to two screensdisplayed in two displays such that a parent screen is displayed in thefirst display and a node screen (e.g., a child screen) is expanded intothe second display. In this regard, each of the screens displayed in thefirst and second display may be independent screens that comprise partof a hierarchical application sequence (e.g., as shown in FIG. 2B).Alternatively, the single screen mode of the multi screen applicationmay simply be scaled such that the contents of the single screen arescaled to occupy both displays. Thus, the same content displayed in thesingle screen is scaled to occupy multiple displays, but no additionalviewing area or graphics are presented. Further still, the maximizationof the multi screen application from a single screen mode to a multiscreen mode may result in the expansion of the viewable area of theapplication. For example, if a multi screen application is displayed insingle screen mode, upon maximization into multi screen mode, theviewable area of the multi-screen application may be expanded while thescale of the graphics displayed remains the same. In this regard, theviewable area of the multi-screen application may be expanded into thesecond display while the scaling remains constant upon expansion.

In this regard, an application may have configurable functionalityregarding the nature and behavior of the screens of the application. Forinstance, an application may be configurable to be a single screenapplication or a multi screen application. Furthermore, a multi screenapplication may be configurable as to the nature of the expansion of themulti screen application between a single screen mode and a multi screenmode. These configuration values may be default values that may bechanged or may be permanent values for various applications. Theseconfiguration values may be communicated to the device (e.g., theprocessor 116) to dictate the behavior of the application when executingon the device.

FIG. 4 depicts various graphical representations of gesture inputs thatmay be recognized by a handheld computing device. The gestures shown aredemonstrative, and as such, other gestures may be provided withoutlimitation. Such gestures may be received at one or more gesture sensorsof the device. In this regard, various input mechanisms may be used inorder to generate the gestures shown in FIG. 4. For example a stylus, auser's finger(s), or other devices may be used to activate a touchsensitive device in order to receive the gestures. Alternatively, thegestures may be detected by an optical device (e.g., a camera). The useof a gesture may describe the use of a truncated input that results infunctionality without the full range of motion necessary toconventionally carry out the same functionality. For instance, movementof screens between displays may be carried out by selecting and movingthe screen between displays such that the full extent of the motionbetween displays is received as an input. However, such animplementation may be difficult to accomplish in that the first andsecond displays may comprise separate display portions withoutcontinuity therebetween. As such, a gesture may truncate the full motionof movement or provide an alternative input to accomplish the samefunctionality. Thus, movement spanning the first and second display maybe truncated so that the gesture may be received at a single touchsensitive device. The use of gesture inputs is particularly suited tohandheld computing devices in that the full action of an input may bedifficult to execute given the limited input and display space commonlyprovided on a handheld computing device.

With reference to FIG. 4, a circle 190 may represent a touch received ata touch sensitive device. The circle 190 may include a border 192, thethickness of which may indicate the length of time the touch is heldstationary at the touch sensitive device. In this regard, a tap 186 hasa thinner border 192 than the border 192′ for a long press 188. In thisregard, the long press 188 may involve a touch that remains stationaryon the touch sensitive display for longer than that of a tap 186. Assuch, different gestures may be registered depending upon the length oftime that the touch remains stationary prior to movement.

A drag 176 involves a touch (represented by circle 190) with movement194 in a direction. The drag 176 may involve an initiating touch thatremains stationary on the touch sensitive device for a certain amount oftime represented by the border 192. In contrast, a flick 178 may involvea touch with a shorter dwell time prior to movement than the drag asindicated by the thinner border 192″ of the flick 178. Thus, againdifferent gestures may be produced by differing dwell times of a touchprior to movement. The flick 178 may also include movement 194. Thedirection of movement 194 of the drag and flick 178 may be referred toas the direction of the drag or direction of the flick. Thus, a drag tothe right may describe a drag 176 with movement 194 to the right.

In an embodiment, a gesture having movement (e.g., a flick or draggesture as described above) may be limited to movement in a singledirection along a first axis. Thus, while movement in a directiondifferent than along the first axis may be disregarded so long ascontact with the touch sensitive device is unbroken. In this regard,once a gesture is initiated, movement in a direction not along an axisalong which initial movement is registered may be disregarded or onlythe vector component of movement along the axis may be registered.

While the directional gestures (e.g., the drag 176 and flick 178) shownin FIG. 4 include only horizontal motion after the initiation of thegesture, this may not be actual movement of the touch during thegesture. For instance, once the drag 176 is initiated in the horizontaldirection, movement in a direction other than in the horizontaldirection may not result in movement of the screen to be moved in thedirection different and the horizontal direction. For instance, withfurther reference to FIG. 7, the drag 176 from left to right may beinitiated with initial movement 204 from left to right along aninitiated direction 210. Subsequently, while the user may input an offdirection movement 206 in a direction different than the initiateddirection 210. In this regard, the off direction movement 206 may notresult in any movement of a screen between two displays. Furthermore,the user may input partially off direction movement 208, where only avector portion of the movement is in the direction of the initiateddirection 210. In this regard, only the portion of the partially offdirection movement 208 may result in movement of a screen betweendisplays. In short, the movement of application screens between thefirst display 102 and the second display 104 may be constrained along asingle axis along which the displays are arranged.

The handheld computing device that is positionable such that a firstdisplay 102 and a second display 104 are both visible from the vantagepoint of the user may allow for expansion of applications to occupy boththe first display 102 and the second display 104. This expansion may beby a variety of techniques such as, for example, by way of receipt of agesture or otherwise. In that applications may include one or morescreens or have screens that may be expanded across multiple displays,it may be advantageous to provide an indication to a user as to whetherthe application screen is expandable, has been expanded, or is in theprocess of expanding.

To this end, a screen may include an icon which indicates to a user thestate of the screen (e.g., be it in an expanded mode, unexpanded mode,or expanding mode). The icon may be morphed along with the expansion ofthe application screen or application into another display. Such anarrangement is depicted in FIGS. 5A-5C. In FIG. 5A, an expandablemultiscreen application 156 is disposed in a first display 102. A firstscreen 156A of the application 156 occupies substantially all of thefirst display 102. An icon 500, is provided on the first screen 156A andmay indicate to a user that the application 156 may be expanded. Theicon 500 may be located along a periphery of the first display 102. Forinstance, as shown, the icon 500 may be arranged along a top portion ofthe display. The icon 500 may be provided adjacent to or superimposed ona status bar 550. As shown, the icon 500 occupies a portion of thestatus bar 550 and the application screen 156A, however otherembodiments may be provided wherein the icon 500 is disposed whollywithin the status bar 550, the application screen 156A, or in some otherfashion. The status bar 550 may provide data such as cellular signalstrength, cellular carrier information, services executing on thedevice, battery strength, time/date information, or other appropriateinformation. In other embodiments, the icon 500 could be provided inother locations such as along the bottom of the first display 102 oraway from a peripheral edge of the display 102.

The icon 500 may have a number of states that correspond with thecondition of the application 156. As such, icon 500 may be in a firststate. In turn, icon 500″ may correspond to the icon 500 in a secondstate (e.g., wherein application 156 is in a second condition). Icon500′ may represent icon 500 in a state between the first state andsecond state.

A second display 104 including a desktop screen 138 is also visible to auser. Upon initiation of expansion of the application 156 into thesecond display 104, shown in FIG. 5B, the icon 500 may morph into icon500′ as the second screen 156B of application 156 begins to expand intothe second display 104. The expansion of application 156 into the seconddisplay may be animated such that the second screen 156B of application156 appears as though it is moving from the first display 102 to thesecond display 104. With this animation, the icon 500 may morph to icon500′ and finally to icon 500″ as shown in FIG. 5C once the application156 has expanded to occupy the first display 102 and the second display104. The icon 500″ may extend across both displays. This mayadditionally indicate to a user that the application 156 may beminimized to occupy only the single display.

In this regard, the icon 500 may include a first dimension and a seconddimension comprising a first aspect ratio. Icon 500″ may have a thirddimension and a fourth dimension comprising a second aspect ratio. Invarious embodiments, the first aspect ratio and second aspect ratio maychange or may remain constant upon expansion of the application 156. Forexample, the first aspect ratio may change as the application 156expands to occupy both displays until finally the icon 500″ is in asteady state, wherein the icon 500″ defines the second aspect ratio. Inthis regard, the first dimension and the third dimension of the icon 500and 500″, respectively, may be the same. That is, icon 500 and 500″ mayshare a common dimension in a first direction. For instance, at thefirst dimension and the third dimension may comprise a height dimension.In this regard, the height of icon 500 may not change as it morphs to500″, however, the width of icon 500 may expand commensurately with theexpansion of the application into the second display. In otherembodiments, the height of the icons 500, 500′, and 500″ may also change(e.g., to maintain a constant aspect ratio); however, such an instanceis not shown.

When modifying the manner in which the application is displayed, themodifying may include a stretching or expanding of the icon 500 into thesecond display 104 in proportion to the expansion of the application156. In this regard, the width (e.g., the first or third dimension) maybe proportionate to the width of the application 156 as it expands intothe second display. That is, the icon 500′ may have a lengthcorresponding to the width of screen 156A and any portion of the widthof screen 1568 as it is expanded into the second display 104. As such,once fully expanded, icon 550″ may have a width equal to the width ofscreens 156A and 1568 combined (e.g., as a continuous icon spanning bothscreens).

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and description isto be considered as exemplary and not restrictive in character. Forexample, certain embodiments described hereinabove may be combinablewith other described embodiments and/or arranged in other ways (e.g.,process elements may be performed in other sequences). Accordingly, itshould be understood that only the preferred embodiment and variantsthereof have been shown and described and that all changes andmodifications that come within the spirit of the invention are desiredto be protected.

What is claimed is:
 1. A method for controlling a handheld computingdevice including one or more displays, comprising: displaying a firstscreen of an application in a first display condition; displaying anicon with the first screen of the application, wherein the iconindicates that the application is in a first state corresponding to thefirst display condition; receiving a gesture input at a gesture sensor;modifying the manner in which the application is displayed in responseto the receiving step; changing the application to a second displaycondition; and morphing the icon to indicate that the application is ina second state corresponding to a second display condition the morphingcomprising changing the shape and size of the icon; wherein the firstand the second states comprise an open application state.
 2. The methodaccording to claim 1, wherein the icon is changed in proportion to therelative position of the application with respect to the first andsecond display.
 3. The method according to claim 2, wherein the firstdisplay condition includes displaying the application in only a firstdisplay, and wherein the modifying includes expanding the application tooccupy at least a first and second display.
 4. The method according toclaim 1, wherein the first display condition includes displaying theapplication in at least a first and second display, and wherein themodifying includes minimizing the application to occupy only one of thefirst and second displays.
 5. The method according to claim 4, wherein afirst and second dimension define a first aspect ratio of the icon and athird and fourth dimension define a second aspect ratio of the icon. 6.The method according to claim 5, wherein the first aspect ratio isdifferent than the second aspect ratio.
 7. The method according to claim6, wherein the first dimension and the third dimension are equal.
 8. Themethod according to claim 7, wherein the first dimension and the thirddimension are vertical dimensions.
 9. The method according to claim 8,wherein the second dimension corresponds to a width of the applicationwhen displayed in the first display, and wherein the fourth dimensioncorresponds with the width of the application when displayed in thefirst and the second display.
 10. The method according to claim 9,wherein the modifying step includes animating the change of theapplication.
 11. The method according to claim 10, wherein the animatingincludes changing of the icon from the first aspect ratio to the secondaspect ratio.
 12. A handheld computing device, comprising: a processor;a first display operable to display a first screen, the first displaybeing in operative communication with the processor; a second displayoperable to display a second screen, the second display being inoperative communication with the processor; at least one gesture sensor,separate from the first display and the second display, and operable toreceive a gesture input; wherein the processor is operable to: executean application that is displayable on at least one of the first displayand the second display in response to the gesture input; and displayingan icon associated with the application in at least one of the firstdisplay and the second display, wherein the icon indicates that theapplication is in one of two or more states associated with a displaystatus of the application with respect to the first and second displays;wherein the first and the second states comprise an open applicationstate.
 13. A device as recited in claim 12, wherein the icon isindicative of whether the application is expandable, whether theapplication is expanded, or whether the application is expanding. 14.The device as recited in claim 13, wherein the handheld computing devicecomprises a smart phone.
 15. The device as recited in claim 14, whereinthe first display and second display are positionable with respect toeach other between an open and closed position.
 16. The device asrecited in claim 15, wherein when in the open position, the firstdisplay and the second display are visible from the vantage point of auser.
 17. The device as recited in claim 16, wherein when in the closedposition, only one of the first display and the second display arevisible from the vantage point of a user.
 18. The method according toclaim 11, wherein the icon indicates one of two or more states.
 19. Themethod according to claim 18, wherein the icon indicates one of threestates, the three states including an expandable state, an expandedstate, and an expanding state.
 20. The method according to claim 12,wherein the icon morphs to indicate that the application is in one oftwo or more states, the morphing comprising changing the shape and sizeof the icon.